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Abstract 

Regional  and  theater  military  operations  have  increasingly  found  open-source  information  to  be  a 
useful  supplement  to  traditional  data  sources.  Open-source  information  includes  a  wide  variety  of 
public  data  that  can  be  readily  acquired  from  news  broadcasts,  web  sites,  on-line  databases, 
information  brokers,  email,  etc.  The  VICTOR  project  is  a  technology  demonstration  initiative 
with  the  objective  of  developing  human-centric  decision  support  principles  for  information 
processing  to  support  on-line  open-source  information  analysis  by  leveraging  existing  /  evolving 
capabilities.  Observations,  interviews,  and  product  assessments  were  conducted  at  USCINCPAC 
to  determine  user  requirements  and  current  work  processes.  Based  on  this,  we  are  developing  a 
set  of  tools  that  are  customized  to  address  the  cognitive  tasks  performed  by  open-source 
information  analysts.  An  innovative  development  approach  is  being  employed  that  provides 
access  to  a  variety  of  relevant  COTS/GOTS  software  through  a  consistent  human-computer 
interface  and  that  adapts  these  tools  to  the  users’  specific  tasks.  This  approach  enables  extremely 
rapid,  low  cost  development  that  stays  abreast  of  the  latest  technology,  while  unburdening  users 
from  excessive  system  integration  and  training. 

1 .  Introduction 

Today’s  military  environment  requires  timely  and  effective  decision-making  in  a  range  of 
peacekeeping  and  war-fighting  situations.  To  provide  decision-makers  with  accurate  information, 
analysts  use  a  number  of  sophisticated  tools  and  techniques  to  gather  information  from  classified 
as  well  as  open-sources.  Open-source  information  includes  a  wide  variety  of  publicly  accessible 
data  that  can  be  acquired  from  news  broadcasts,  web  sites,  on-line  databases,  information  brokers, 
email,  etc.  Unfortunately,  many  of  the  tools  and  techniques  currently  in  use  for  this  purpose  are 
inadequate  for  military  tasks,  difficult  to  use,  or  lack  the  required  level  of  integration  and 
compatibility.  Worse  yet,  some  needed  tools  are  technologically  immature  or  simply  do  not  exist. 
As  a  result,  vital  information  is  often  inaccessible  to  the  decision-maker. 
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SPAWAR  Systems  Center,  San  Diego  (SSC-SD),  under  ONR  sponsorship,  is  currently 
conducting  the  Virtual  Information  Center  Technologies  for  Open-Source  Requirements 
(VICTOR)  Project  to  address  these  special  information  management  requirements.  Extensive 
efforts  have  been  made  to  understand  the  user  community  and  their  needs,  the  tasks  they  perform, 
the  resources  and  tools  they  use,  and  the  products  they  generate.  From  this,  we  identified 
important  issues  and  problem  areas.  A  human-computer  interface  (HCI)  design  concept  and 
demonstration  were  then  developed  highlighting  a  representative  user’s  performance  of  routine 
tasks  using  the  improved  tools.  Using  the  HCI  design  as  a  starting  point,  we  have  been 
developing  a  tool  set  to  facilitate  the  identification,  collection,  fusion,  analysis,  management,  and 
presentation  of  open-source  data. 

2.  User  Requirements  and  Work  Processes 

The  primary  customer  of  the  VICTOR  Project  efforts  is  the  Virtual  Resource  Center  (VRC) 
located  at  USCINCPAC.  Staffed  by  a  group  of  highly  skilled  military  and  civilian  researchers  and 
analysts,  the  VRC  collects  and  provides  open-source  information  needed  by  US  decision-makers. 
As  the  VICTOR  tool  set  and  related  concepts  mature,  its  products  are  intended  to  transition  to 
other  organizations  with  similar  needs. 

The  first  step  in  the  human-centric  decision  support  process  used  here  is  to  understand  the  users’ 
background,  needs,  tasks,  resources,  and  products.  To  do  this,  we  observed  VRC  personnel  in 
situ ,  interviewed  VRC  staff  about  their  tasks  and  resources,  and  analyzed  the  format  and  content 
of  their  projects.  We  also  interviewed  several  “consumers”  of  VRC  products  to  determine  how 
well  these  products  met  their  needs  for  open-source  information. 

2.1  Open-Source  Information  Products 

The  VRC  generates  several  types  of  open-source  information  products.  These  include:  (a) 
abstracts  and  summaries  of  relevant  current  news  stories,  (b)  primers  that  compile  a  wide  range  of 
relevant  information  about  a  country  or  region,  (c)  research  reports  that  predict  long-term  trends, 
(d)  a  database  of  frequently  asked  questions  about  the  countries  and  regions  within  the  Pacific 
theater,  and  (e)  replies  to  specific  requests  for  information  from  anyone  in  the  command. 

One  dimension  on  which  the  VRC  products  differ  is  in  the  amount  of  situational  context.  Routine 
information  searches  occur  when  little  context  is  available,  while  tailored  searches  occur  in  high 
context  situations.  Another  dimension  that  characterizes  VRC  products  concerns  how  they  were 
initiated.  Some  products  are  reactive  to  requests  for  information  from  outside  the  VRC,  while 
others  are  initiated  by  the  VRC  proactively. 

Table  1  illustrates  this  representation  of  VRC  products.  Products  in  the  first  cell  (routine  - 
reactive)  result  from  implicit  queries,  where  a  standing  request  triggers  the  search  on  a  regular 
basis.  Products  in  the  next  cell  (routine  -  proactive)  emerge  from  the  news  stories  reported  each 
day.  Tailored  search  products  can  be  seen  as  arising  from  explicit  queries  (e.g.,  RFIs)  or  from  an 
anticipation  of  decision-maker  information  needs  (e.g.,  primers). 


Table  1.  Representation  of  VRC  Products 


Routine  Search 
(Low  Context) 

Tailored  Search 
(High  Context) 

Reactive 

Implicit  Queries 

Explicit  Queries 

e.g.,  Frequently  Asked  Questions 

e.g.,  Requests  for  Information 

Proactive 

Emergent  Queries 

Anticipatory  Queries 

e.g.,  News  Abstracts  /  Summaries 

e.g.,  Primers,  Research  Reports 

2.2  VRC  Staffing 

These  products  can  be  developed  by  several  geographically  distributed  groups,  linked  together  by 
an  information  network.  This  “virtual  information  center”  includes  the  Center  of  Excellence  -  for 
connections  with  non-governmental  and  private  volunteer  organizations,  the  Pacific  Disaster 
Center  -  for  weather  and  oceanographic  information,  commercial  information  brokers  -  for  in- 
depth  reports  on  selected  topics,  and  the  VRC  at  USCINCPAC. 

Within  the  VRC,  there  are  several  military  and  civilian  staff  with  varying  degrees  of  experience 
with  the  open-source  information  analysis  tasks.  The  Researchers,  many  of  whom  are  reservists, 
use  Internet  search  tools  to  gather  basic  open-source  information  such  as  news  articles.  The 
Analysts,  then,  use  the  information  collected  by  the  Researchers  to  build  or  adapt  a  product.  The 
VRC  Director  oversees  the  work  assignments  of  the  Researchers  and  Analysts  and  reviews  the 
final  products  before  they  are  forwarded  or  posted.  In  addition,  several  software  engineers  and  a 
system  administrator  support  the  VRC’s  operation. 

2.3  Work  Processes  and  Cognitive  Demands 

While  generating  the  VRC  products  involves  different  resources  and  procedures,  there  are  five 
general  functions  that  are  common  to  all:  (a)  structure  the  query  /  topic,  (b)  acquire  the  data,  (c) 
organize  the  data,  (d)  interpret  the  data,  and  (e)  produce  the  report.  Work  flow  diagrams  were 
created  to  characterize  the  detailed  steps  within  each  of  these  functions  that  are  necessary  to 
generate  each  type  of  VRC  product,  to  identify  resources  and  tools  currently  used,  and  to  indicate 
how  information  is  exchanged  among  staff. 

Particular  emphasis  was  focused  on  the  tasks  that  were  the  most  cognitively  demanding  and  that 
continue  to  pose  the  greatest  challenge  for  VRC  staff.  Two  of  these,  structuring  queries  and 
summarizing  retrieved  data,  are  discussed  in  terms  of  their  general  requirements  for  the  VICTOR 
tool  set. 

2.3.1  Structuring  Queries 


It  is  especially  difficult  to  determine  what  topics  and  sources  to  search  when  a  query  for  open- 
source  information  comes  from  someone  outside  the  VRC.  Queries  phrased  in  natural  language 


are  easily  and  often  misinterpreted,  and  the  underlying  goal  of  the  request  is  not  easy  to  discern. 
The  VRC  researcher  /  analyst  must  determine  the  context  and  purpose  of  the  request,  the  intended 
use  of  the  information,  information  that’s  already  known,  the  deadline  for  the  report,  the  precision 
required,  etc.  In  addition,  it  is  often  helpful  to  get  a  starting  place  for  the  search,  which 
sometimes  may  be  a  previous  search,  archived  information,  or  even  a  good  web  site. 

2.3.2  Summarizing  Information 

Following  an  Internet  search  on  a  topic,  a  huge  number  of  web  sites  and  articles  are  typically 
retrieved,  many  of  which  are  conflicting  or  redundant.  For  example,  in  searching  for  information 
related  to  the  query,  “How  many  casualties  might  be  expected  following  a  major  volcanic  eruption 
in  Indonesia?”  we  obtained  over  330,000  potentially  useful  resource  sites.  Even  though  only  a 
few  hundred  sites  may  be  directly  relevant,  many  of  these  are  contradictory,  uninformative,  or  not 
current.  There  is  a  need  to  summarize  massive  quantities  of  data  into  a  single,  concise  answer. 
Summaries,  however,  should  still  indicate  the  quantity,  reliability,  location,  and  perishability  of  the 
data,  should  enable  users  to  directly  access  the  original  data  if  desired,  and  should  allow  users  to 
specify  the  amount  of  detail  provided.  To  reduce  the  impact  of  decision  biases,  supporting, 
conflicting,  missing,  and  related  information  should  be  presented  together  and  significant 
assumptions,  inferences,  and  “leaps  of  faith”  should  be  made  explicit. 

3.  Prototype  VICTOR  Tools 

Several  human-computer  interface  (HCI)  functional  requirements  have  emerged  from  this 
analysis.  First,  the  VICTOR  tool  set  must  be  easy  to  learn  and  easy  to  use.  The  large  number  of 
inexperienced,  short-term  staff,  the  high  workload,  and  the  rapid  deadlines  in  the  VRC  make  this 
requirement  a  high  priority.  Second,  the  VICTOR  tool  set  must  be  powerful,  taking  advantage  of 
state-of-the-art  information  technology  in  order  to  retrieve  hard-to-find  data  without 
overwhelming  users  with  redundant  or  irrelevant  hits.  Next,  the  tools  must  be  flexible,  since  each 
mission  is  likely  to  require  a  different  type  of  tools,  methods,  and  data.  Finally,  the  VICTOR  tool 
set  must  be  modular  in  order  to  stay  abreast  of  the  latest  developments  in  the  many  related  areas 
of  information  technology. 

The  prototype  VICTOR  tools  are  discussed  in  detail  and  demonstrated  as  part  of  this 
presentation.  Figure  1  illustrates  how  these  prototype  tools  relate  to  each  other  and  fit  with  the 
VRC’s  work  flow. 
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Figure  1 .  Prototype  VICTOR  tools  that  support  open-source  information  analysis. 

3.1  Query  Wizard 

A  simple,  interactive  “interview  agent”  helps  requestors  and  VRC  analysts  phrase  and  parse  open- 
source  information  requests.  This  helps  to  disambiguate  commonly  used  words,  establish  the 
context  for  the  request,  identify  links  to  other  requests,  and  determine  the  urgency  of  the  request. 
The  interview  agent  can  also  help  construct  multiple  specific  requests  from  a  general  query.  For 
example,  the  general  query,  “What  would  be  the  effects  of  a  major  volcanic  eruption  in 
Indonesia?”  might  be  parsed  into  separate  queries  focusing  on  the  impact  on  the  local  population, 
the  regional  climate,  the  food  supply,  and  the  long-term  economy. 

3.2  Workflow  Management 

This  tool  helps  the  VRC  staff  to  stay  informed  about  upcoming  events,  task  priorities,  and  status 
of  ongoing  analyses.  It  provides  a  useful  focal  point  for  updates  and  alerts  about  the  latest  world 
events  and  news  that  may  be  relevant  to  military  concerns.  The  Director  can  also  use  this  tool  as 
a  way  to  assign  tasks  to  personnel,  monitor  workload  of  the  staff,  and  to  reset  task  priorities. 

3.3  Data  Collection 

Many  excellent  search  engines  and  agents  are  available  commercially  to  support  the  VRC’s  need 
to  collect  open-source  data  on  a  wide  range  of  topics.  Rather  than  replicate  any  of  these,  this 
VICTOR  tool  provides  users  with  a  portal  from  which  to  access  these  commercial  tools. 
However,  the  VICTOR  Super  Search  tool  provides  a  common  HCI  and  makes  it  easy  for  users  to 


specify  the  breadth,  depth,  and  means  of  their  searching.  This  tool  is  designed  to  serve  the 
searching  needs  of  both  experts  and  novices. 


3.4  Data  Visualization,  Summarization,  Integration,  and  Reporting 

VICTOR  is  developing  a  series  of  tools  to  help  users  to  compile,  organize,  and  visualize  the 
diverse  forms  of  open-source  data  that  are  typically  retrieved.  A  key  feature  of  this  tool  is  its 
ability  to  present  multi-media  data  in  common  workspaces  to  promote  fusion  and  integration.  As 
analysts’  may  require,  other  tools  and  simulations  can  be  consulted  via  web  links  in  order  to  help 
interpret,  disambiguate,  or  forecast  patterns  in  the  retrieved  data.  Throughout  this  mediated 
analysis  process,  VICTOR  is  building  a  draft  of  the  VRC  report  that  fits  structural  templates 
customized  for  the  kind  of  information  needed  by  various  types  of  open-source  information 
consumers  in  the  command. 

4.  Leveraged  Development 

Many  of  the  VICTOR  tools  are  intentionally  similar  to  commercial  software  products  that  are  in 
common  usage  at  most  offices,  including  the  VRC.  In  fact,  our  design  and  development  approach 
explicitly  attempts  to  leverage  these  strengths  to  support  our  application.  VICTOR  tools  provide 
a  customized  version  of  these  applications,  with  pre-specified  templates  and  links  to  other  useful 
resources.  There  are  many  clear  advantages  to  this  leveraged  development  approach,  including 
lower  software  development  cost  and  time,  quicker  alignment  with  state-of-the-art  commercial 
software  applications,  greater  compatibility  with  tools  used  for  other  purposes,  and  reduced 
burden  for  training  and  system  integration.  The  most  significant  challenge  with  this  approach  is  to 
determine  which  applications  best  fit  the  user’s  task  requirements  and  to  develop  an  appropriate 
and  usable  common  HCI  from  which  users  can  access  these  applications.  This  is  why  the 
VICTOR  project  has  adopted  the  human-centric  decision  support  process  that  begins  with  a  clear 
understanding  of  the  users’  tasks,  products,  resources,  and  workflow. 

5.  Ongoing  Efforts 

The  VICTOR  project  is  still  in  the  early  stages,  but  already  several  prototype  technologies  have 
been  installed  at  the  VRC  in  USCINCPAC  for  evaluation.  Feedback  from  these  prototypes  has 
helped  guide  further  development  efforts  to  refine  and  expand  the  tools  provided  for  open-source 
information  analysis.  Our  particular  focus  over  the  next  year  will  be  on  supporting  the  analysis 
and  reporting  of  large  amounts  of  conflicting  open-source  data  using  multi-media  data 
visualization  and  conceptual  structuring  tools. 

This  spiral  design-development-testing  process  has  also  been  quite  beneficial  for  the  more  basic 
research  aspects  of  the  project.  By  providing  operational  users  with  an  initial  version  of  some 
advanced  information  technology,  we  can  determine  whether  and  how  they  might  actually  use  it 
and  what  pre-  or  post-processing  may  be  required.  This  enables  us  to  identify  promising 
technologies  rapidly,  without  a  huge  investment  in  software  development.  At  the  same  time,  this 
user  feedback  on  prototypes  has  identified  important  unresolved  questions  where  more  basic 
research  is  needed  through  human  factors,  cognitive  science,  and  team  collaboration  studies. 


